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The United States Medical Licensing Examination 
(USMLE) Step 1 was recently made pass/fail. This deci- 
sion was controversial largely because of the reliance on 
USMLE Step 1 scores in resident selection. However, 
these scores do not correlate with resident ability. In 
this manuscript, we consider if the American Board of 
Surgery In-Training Examination (ABSITE) should be 
pass/fail as well. The ABSITE has been used for “high- 
stakes” purposes, such as preliminary resident advance- 
ment and prospective fellow evaluation, for which it 
was not intended. Moreover, similar to the USMLE Step 
1 exam, ABSITE scores have demonstrated no correla- 
tion with clinical ability. A pass/fail ABSITE would return 
the exam to its original purpose and minimize an over- 
reliance on scores. Moving forward, new objective meas- 
ures will need to be developed to assess surgical trainees 
in a more holistic manner. (J Surg Ed 000:1—3. © 2020 
Association of Program Directors in Surgery. Published 
by Elsevier Inc. All rights reserved.) 
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BACKGROUND 


The United States Medical Licensing Examination 
(USMLE) Step 1 was recently made pass/fail. This move 
retains the primary purpose of the exam, determining 
medical licensure eligibility,’ but came after lengthy 
deliberation. A key reason for the change was to reduce 
the perceived overemphasis on the exam, especially in 
residency selection.” 
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Despite being widely used in residency selection, 
USMLE Step 1 scoring has not been shown to correlate 
with overall resident ability or success,* perhaps with 
the exception of extremely low scores that have been 
found to correlate with an increased risk of failing the 
boards.‘ Programs sometimes rely on this score because 
it is the only national objective yardstick, however it is 
not a perfect measure.” USMLE Step 1 assesses medical 
knowledge well, but this is only one of six core compe- 
tencies endorsed by the Accreditation Council for Gradu- 
ate Medical Education.’ A well-trained physician is 
expected to embody all other core competencies: 
patient care, practice-based learning and improvement, 
interpersonal and communication skills, professional- 
ism, and systems-based practice. 


THE AMERICAN BOARD OF SURGERY IN- 
TRAINING EXAMINATION 


The American Board of Surgery In-Training Examination 
(ABSITE) is taken yearly by all general surgery trainees in 
the United States. It was designed to be a “low-stakes” 
check-in and marker of medical knowledge. The Ameri- 
can Board of Surgery (ABS) does not use the score for 
any purpose; they simply report the score to programs. 
In fact, board certification is not contingent on ABSITE 
performance. The intention of the exam is to allow indi- 
vidual programs to use deficient scores to design remedi- 
ation plans. 

In a pattern analogous to the use of the USMLE Step 1, 
the ABSITE has developed into a “high-stakes” exam. Pro- 
grams have sometimes used the exam as a factor for pro- 
motion to the next Postgraduate Year level.° Though 
this is far less commonplace than in the past, the exam is 
still frequently relied upon in the selection of prelimi- 
nary resident advancement. For preliminary residents, 
exam performance may be “life or death,” as it is fre- 
quently a key factor in securing a categorical position. 
The exam has also become an important metric in the 
evaluation of prospective fellows.’ Though there is no 
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minimum ABSITE threshold, and there are other consid- 
erations, the ABSITE is a critical metric when screening 
for who to interview and is a factor in evaluating appli- 
cants.’ In a survey of pediatric surgery fellowship pro- 
gram directors in 2016 the only factors that had 
universal consensus as important for granting interviews 
were residency institution and ABSITE scores.” 

The ABSITE attained such importance because many 
believed it to be a predictor of program ABS board exam 
pass rates.” However, the data on this is mixed." Jones 
et al. demonstrated that a higher ABSITE score may be 
predictive of passing the ABS Qualifying Exam (QE) on 
the first attempt, but found no such correlation with the 
ABS Certifying Exam (CE).° Furthermore, they were 
unable to demonstrate a significant correlation between 
low ABSITE scores and QE failure. 

Regardless of the ABSITE’s ability to predict board 
exam success, using the ABSITE to such a great degree 
to decide on fellow selection appears to be flawed in the 
same manner of using the USMLE Step 1 for resident 
selection. It creates an unbalanced assessment of surgi- 
cal trainees that focuses heavily on medical knowledge, 
while ignoring other important core competencies. 

No correlation has ever been demonstrated between 
ABSITE scores and clinical ability, as measured by faculty 
evaluations.'”'* As such, it is likely not prudent to rely 
heavily on ABSITE scores for preliminary resident 
advancement or prospective fellow evaluation. Some 
programs have had residents repeat a Postgraduate Year 
level due to poor ABSITE performance alone.'* This may 
not happen regularly, as it did in the past. Some resi- 
dents may ultimately benefit from repeating the year, 
but an academic remediation plan with structured study 
time may be a better alternative. Daily reading time has 
been found to correlate well with ABSITE success. '“ 

There is a lack of data to support the reliance on 
ABSITE scores for these “high-stakes” decisions. It is 
likely that many have come to rely on the ABSITE 
because it is an objective measure in a world deficient of 
other objective measures. However, the mere presence 
of an objective measure does not make it a good one. 


THE ARGUMENT FOR A PASS/FAIL ABSITE 


The ABSITE has deviated from its initial purpose as a 
“low-stakes” exam. In this process it has become a mea- 
sure for many “high-stakes” decisions, but it is not neces- 
sarily a good measure for these purposes. A transition to 
a pass/fail ABSITE could represent a return to the exam’s 
original purpose and help distance surgical education 
from an over-reliance on scores. It would, as the USMLE 
program wrote, reduce the overemphasis on the exam, 
while retaining the primary purpose of the exam. 


A pass/fail test would accomplish the original goal of a 
“low-stakes” medical knowledge check in. It would still 
lend itself well to the setup of remediation pathways. It 
may relieve some of the stress of surgical residency and 
potentially lead to better exam performance. As a bonus, 
residents may feel less compelled to attend ABSITE review 
courses, which can cost upwards of $1,000 and have not 
demonstrated efficacy." This change would eliminate the 
unnecessary and poor “high-stakes” purposes for which 
the exam has been used. Furthermore, this change could 
help push the surgical community to more holistically eval- 
uate trainees for advancement or fellowship selection. 

An important question is the pass/fail threshold. Many 
surgical training programs use the 20th percentile as a 
standard pass rate because the ABS QE has approxi- 
mately a 20% fail rate. However, given the uncertainty of 
correlation between the 2 exams, we propose a more 
conservative pass number, the 10th percentile. In our 
view, a fail would not be associated with any additional 
consequences — specifically, not failing the clinical year. 
The fail would merely provide the revelation that the res- 
ident needs to improve their fund of knowledge. 


THE ARGUMENT AGAINST A PASS/FAIL 
ABSITE 


Of course, there are also reasons why a pass/fail ABSITE 
may not be beneficial. Reasons to maintain the status 
quo include the potential for decreased clinical knowl- 
edge. A pass/fail structure may remove the incentive to 
study. Although this is possible, we bank on the inherent 
drive to succeed that led people to surgical residency in 
the first place. Furthermore, reading can be encouraged 
in other ways including developing reading schedules, 
issuing quizzes on reading material, or organizing discus- 
sions on reading assignments. Internal competition with 
question banks and reading challenges may be more 
effective in stimulating regular reading habits. 

It is true that a lack of other objective measures could 
be problematic. However, using this objective measure 
just because it is available is not necessarily better. Other 
objective measures are needed whether ABSITE is made 
pass/fail or not. 


FUTURE IMPLICATIONS 


Despite the ABSITE being a poor overall assessment of a 
surgical trainee, a transition to a pass/fail test would be 
followed by the question of how to assess trainees. Fac- 
ulty evaluations would be a reasonable place to start for 
a holistic evaluation. Tarpley et al. found that faculty 
evaluations are a more global assessment of competency 
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relative to the ABSITE.'? We propose the development 
of a standardized evaluation system that could be used 
by residency programs. Other objective measures will 
need to be sought out, but it will be imperative that 
these measures represent the breadth of the core compe- 
tencies. Currently, this is best addressed by the program, 
not the single medical knowledge metric. 
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